Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: implement fast node #75

Merged
merged 4 commits into from
Apr 25, 2024
Merged

Conversation

yutianwu
Copy link
Collaborator

@yutianwu yutianwu commented Mar 26, 2024

Description

Currently a op-geth node has two kinds of state world: MPT and snapshot. MPT(Merkle Patricia Tries) is a tree-structured state world. The key function of MPT is to generate the state root to ensure state consistency, while the query/commit on MPT is quite slow. Snapshot is a flattened key-value-based state world. Snapshot provides fast queries and commits. The storage size of the snapshot increases slowly even with a large transaction volume. Snapshot is usually used for block processing, while MPT is used for state verification.

In order to lower the hardware requirement, we introduce the fast node. The fast node will do block processing with snapshot, it will do all verification against blocks except state root.

The fast node doesn’t need to store MPT, so the storage and computation requirement will be much lower.

Since the fast node don'st store the MPT, we can't verify the state root mentioned above. We called it insecure-no-tries. But it should be reliable since we compared the diff layers of the fast node and normal node, it's identical.

It's only encouraged to use the fast node for the normal rpc nodes to reduce costs and improce the syncing performance.

Command
  1. Fast Node

If you want to enable the fast node, you can add --allow-insecure-no-tries in the command.

 ./geth --config ./config.toml --datadir ./node --syncmode full  --allow-insecure-no-tries
  1. Prune

If you want to prune the MPT state, you can also prune the node:

./geth snapshot insecure-prune-all --datadir ./datadir ./genesis.json

Performance

image

We compared the performance of the fast node and normal node by starting two nodes in full sync mode from the genesis of mainnet.

You can refer to the above image, the fast node finish the sync before the normal node more than two days.

Rationale

To lower the cost of the geth nodes and improve the syncing performance.

Example

N/A

Changes

Notable changes:

  • add --allow-insecure-no-tries in command line
  • add insecure-prune-all command to prune MPT state

@github-actions github-actions bot requested review from owen-reorg and redhdx March 26, 2024 09:30
@yutianwu yutianwu changed the title [WIP] implement fast node implement fast node Apr 7, 2024
@yutianwu yutianwu force-pushed the fastNode branch 2 times, most recently from 40e4ecb to c2b6a2e Compare April 7, 2024 09:05
cmd/geth/snapshot.go Outdated Show resolved Hide resolved
cmd/utils/flags.go Outdated Show resolved Hide resolved
core/blockchain_reader.go Outdated Show resolved Hide resolved
go.mod Show resolved Hide resolved
@owen-reorg owen-reorg requested a review from welkin22 April 9, 2024 06:29
@yutianwu yutianwu changed the title implement fast node feat: implement fast node Apr 10, 2024
unclezoro
unclezoro previously approved these changes Apr 10, 2024
owen-reorg
owen-reorg previously approved these changes Apr 24, 2024
redhdx
redhdx previously approved these changes Apr 24, 2024
Copy link
Contributor

@redhdx redhdx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

core/blockchain.go Show resolved Hide resolved
core/state/statedb.go Show resolved Hide resolved
core/state/statedb.go Show resolved Hide resolved
core/state/statedb.go Show resolved Hide resolved
core/state_processor.go Show resolved Hide resolved
@owen-reorg owen-reorg dismissed stale reviews from redhdx and themself via 16bd492 April 25, 2024 02:23
Copy link
Contributor

@welkin22 welkin22 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@owen-reorg owen-reorg merged commit 882e17a into bnb-chain:develop Apr 25, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants